Low rank approximation of polynomials 

Alexander SchrijveiQ 

Abstract. Let k < n. Each polynomial p G R[xi, . . . , x n ] can be uniquely written as p = ^2 /ip^, 
where \i ranges over the set M of all monomials in R[xi, . . . , Xfc] and where p^ G R[xfc+i, . . . , x„]. If 
p is d-homogeneous and e > 0, we say that p is s- concentrated on the first k variables if 

V max p^(x) 2 < e\\p\\ 2 , 
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where \\p\\ is the Bombieri norm of p. We show that for each d G N and e > there exists kd, s 
such that for each n and each (^-homogeneous p G R[xi, . . . ,x n ] there exists k < kd, e such that p 
is e-concentrated on the first k variables after some orthogonal transformation of MP. (So fcd,e is 
independent of the number n of variables.) 

We derive this as a consequence of a more general theorem on low rank approximation of poly- 



q! 1. Introduction 

^ . A polynomial is said to have rank 1 if it is nonzero and a product of linear functions. The 

\ rank of a polynomial p, denoted by rank(p), is the minimum number of rank 1 polynomials 

that add up to p. Low rank of polynomials helps in computing them. Therefore, we 
investigate in how much polynomials can be approximated by low rank polynomials. In 
particular, we consider cf-homogeneous polynomials (i.e., homogeneous polynomials of total 
degree d). 

In fact there are a number of different notions of rank function of a polynomial, but for 
what follows we can take any of them, as long as the function is invariant under orthogonal 
transformations of the polynomial and it does not increase if we set variables in the polyno- 
mial to 0. So one can also take the minimum number of linear functions that generate p, or 
the Waring rank: the minimum number of powers of linear functions that linearly span p. 
| Also one may take the minimum number of rank 1 polynomials of Bombieri norm at most 

1 that add up to p (for the definition of Bombieri norm see below). More strongly, one can 
take any monotonically nondecreasing function of any rank function. 

We recall the Bombieri norm \\p\\ of a d-homogeneous polynomials p G M.[x\, . . . , x n \: 

(1) blh=(E LlaJ ^) V2 > 

oGN n 

where, for a G N n , p a be the coefficient of x" 1 • • • x" n H 
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2 The Bombieri norm is the tensor norm if we view d-homogeneous polynomials as symmetric d-tensors. In 
our estimates we can replace the Bombieri norm equivalently by the square root of the sum of the squares of 
the coefficients, as we fix d and as the two values are bounded by each other up to a factor of d\. However, the 
Bombieri norm behaves better algebraically — in particular, it is invariant under orthogonal transformations 
of R". 
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Let us remark that it is not true that for each d 6 N and e > there is a k £ N such 
that for each ci-homogeneous polynomial p there exists a d-homogeneous polynomial q of 
rank at most k such that ||p — q\\ < e\\p\\. This is shown by the polynomials X^=i x l- 

However, the approximation becomes valid by considering the operator norm: 



(2) ||p|| := max \p{x)\. 



Then Fernandez de la Vega, Kannan, Karpinski, and Vempala [1] showed (it is also a special 
case of the 'weak regularity for Hilbert spaces' in Lovasz and Szegedy [2]) 

(3) for each d € N and e > there exists k € N such that for each ci-homogeneous 

polynomial p (in any number of variables) there exists a d-homogeneous polyno- 
mial q of rank < k such that \\p — q\\ Q < e\\p\\ for each x. 

Important in ([3]) is that k is independent of the number of variables. In fact, in [1] it is 
shown that one can take k = L £ ~ 2 J • 

In this paper we give an extension of ([3]), using the compactness result of [3] as main 
tool. 



2. Stronger low rank approximation 

To describe the extension, let for any n- variable polynomial p and any subspace V of R n , iry 
denote the orthogonal projection onto V, and py := poity. Then ||p|| may be alternatively 
described as the supremum of ||pv|| taken over all 1-dimensional subspaces of R n . As an 
extension, define for any k £ N: 

( 4 ) \\p\\(k) : = sup \\py\\. 

V subspace 
dim V<k 

So ||p||o = |H|(i)- Moreover, as \\py\\ < \\p\\ for any subspace V, we have < ||p||. 

Let P be the set of all ci-homogeneous polynomials in R[xi,X2, • • •], each using only a 
finite number of variables. Call a function r : P — > R monotone if it is invariant under 
orthogonal transformations of the space and if it does not increase if we set variables in a 
polynomial to 0. Each of the above rank functions is monotone, and also each of the above 
norms. Moreover, if r is monotone and / : R —> R is monotonically increasing, then also 
/ o r is monotone. 

Then in (J3]), one may replace \\p— q\\ Q by \\p— q\\ /( ran k(<j))> f° r an Y fixed function / : N — > N 
given in advance. In fact: 

Theorem 1. For each d € N, e > 0, and monotone r : P N, there exists kd, e ,r such that 
for each p G P there exists q G P with r{q) < kd t£; r and \\p — <2l|(r(g)) — £ \\p\\- 

Proof. Let d € N, e > 0, and r : P — > N be given. As P has an inner product associated 
with the Bombieri norm ||.||, the completion H of P is a Hilbert space. As for each k, 
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H-H(fc) < 11-11) the norm ||.||(fc) extends uniquely to H. Let B(P) and B(H) be the closed unit 
balls of P and H respectively. 

Then for each fixed k, the norm ||.||(m is continuous with respect to the ||.|| -topology 
on H. This follows from the fact that there exists a c such that ||p||(fe) < c||p|| for each 
p G P. Indeed, consider any subspace V of dimension k, which we may assume to be M k . 
Now the collection G of all d- homogeneous polynomials p G M fc with ||p|| < 1 is bounded. 
Otherwise, as G is a compact convex set in finite dimensions, there would be a nonzero p 
with ||p||o = 0. However, if p ^ 0, then p(x) / for some x. Hence there exists a c such 
that ||p|| < c for all p G G, which is c as required. 

Define, for each q G P, 

(5) [/, := {p G B(H) | \\p - q\\ {r{q)) < e}. 

So C/ ? is open in the ||.|| -topology. Moreover, the t/q for q G B(P) cover B(H). Indeed, for 
any p G there exists g G -B(-P) with ||p — q\\ < e. Then ||p — <?||( r ( 5 )) < ||p — q\\ < £, so 

Let G be the group of all transformations that consist of an orthogonal transformation 
of W l for some n, leaving the other coordinates invariant. Then G acts naturally on H. By 
[3], the orbit space (B(H), \\.\\ )/G is compact. (This is the quotient topological space of 
the topological space (B(H), ||.|| ), taking the orbits of G as quotient classes.) Hence there 
is a finite set Q C B(P) such that voor each p G B(P) there exist q G Q and ip G G such 
that p^ G U q . Let k^ £ ,f '■= max{r(g) | q G Q}. We show that k^ £ j is as required. 

Let p G P, say p G R[xi, . . . ,x n ]. We may assume that \\p\\ = 1. Then there exist 
q £ Q and V € G such that p^ G ?7 ? . So ||p^ — 9||( r ( g )) < £■ As ||-||( r ( g )) is G-invariant, 
this gives, setting </? := V' -1 ) Hp ~~ ^llfrfg)) < £ - Let </ be the orthogonal projection 
of q^ onto R[xi,...,x n ]. So r(g') < r(^) = r(q) < k d>£ j. Then \\p - q'\\( r (q')) < s, 
for let U be a subspace of W n of dimension < r(q'). Then ((/)[/ = {q^u, and hence 

ll(p - q%\\ = \\(p- <P)u\\ < \\p - q^krw)) < e. I 
3. e- concent rat ion on first k variables 

Let p G R[xi, . . . , x n ] be (i-homogeneous, and let k < n. For a G N fc , denote x a := 
x i 1 ' ' ' x k k ■ Then p can be uniquely written as 

(6) p= Yl xCXp °" 

aeti k 
\ a \=d 

where p a G W l ~ k and \a\ := a\ + ■ ■ ■ + We say that p is e- concentrated on xi, . . . , Xk if 

(7) Yl b-llo<*ll- 

aSN fe 
|a|<d 
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Corollary la. For each d and e > there exists k^ e such that for each n and each d- 
homogeneous p € R[xi, . . . ,x n ] there exist k < kd,e and an orthogonal transformation tp of 
M. n such that p^ is e- concentrated on the first k variables. 

Proof. Let / : N — > N be defined by f(k) := k+ the number of ^-variable monomials 
of degree < d. For any (i-homogeneous polynomial q, let w(q) be its Waring rank (the 
minimum number of <i-powers of linear functions that span q), and Let r{q) := f(w(q)). 
Let kd : e '■= &d, £ /d!,r> where the latter is taken from Theorem [TJ We show that fc^e is as 
required. 

Let p S M.[x\, . . . ,x n ], and let q € P be as given by Theorem [TJ Then k := w(q) < 
f(q) < kd t£ = kd,s/dl,r- As Q nas Waring rank k, there exists a /c-dimensional subspace U of 
R n such that q = qu- By applying an orthogonal transformation of R n , we can assume that 
U = R k . So q <E R[ xi, . . . j Xfc]. We prove that p satisfies ([7]). 

For each a £ N fc with [a| < d, choose z a G M. n ~ k with ||z a || = 1 maximizing |p a (z a )|. 
Let V be the space spanned by R fc and by the z a . Then \\p a \\ = \\Pa(za)\ < |(pv)«|| = 
II Cpv ~~ Pc/)a|| f° r each such a. Moreover, dim(V) < f(k) = f(w(q)) = r(q). So 

(8) £ wis < E life* -^n 2 = E E ua_J"V -Pir)^ < 

|a|<d \a\<d \a\<d |/3|=d-|a| 

rf! E Lf.,J Zl E (aSLJ'V - ^ = di\\Pv-Puf < 

|a|<d |S|=d-|a| 

d!||pv " 9II 2 = dl\\(p - q)v\\ 2 <d\\\p- q\\l, {q)) < S 2 \\p\\ 2 . 

Here \\py — Pu\\ < \\pv ~ q\\ follows from the fact that q = qu and pu = (pv)u, so pu is the 
polynomial defined on U closest to py . I 

Let ||p||oo be the maximum absolute value of the coefficients of p. (This norm is not 
invariant under orthogonal transformations.) We note that for any d there is a c such that 
IHloo < c||p|| for each d-homogeneous polynomial. This follows from the facts that ||p||oo = 
||py||oo for some d-dimensional subspace V of R n and that the set C of (i-homogeneous 
polynomials in R[xi, . . . , xj] with ||p|| < 1 is bounded. This implies that in Corollary [Tal 
one may replace ||p a ||o by II Pa 1 1 oo • 
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